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SUMMARY 

Situational  awareness  plays  a  major  role  in  many  military  and  civilian  operations.  Apart  from  the  identity,  type, 
location  and  dynamics  of  targets  of  interest,  the  situational  picture  may  also  provide  other  target  information, 
such  as  weapons  state,  fuel  status  and  intent.  Many  legacy  systems  incorporate  an  automatic  tracking  capabil¬ 
ity,  with  identification,  situational  assessment  and  decision-making  being  left  to  the  operators.  The  automation 
of  many  of  these  functions  is  the  focus  of  much  research  and  development. 

A  necessary  prerequisite  for  updating  the  state  of  a  target  is  the  correct  association  of  measurements  or 
other  information  to  the  track.  The  ability  of  Bayesian  belief  networks  (BBNs)  to  model  the  uncertain  rela¬ 
tionships  between  continuous  and  discrete  variables  make  them  excellent  candidates  for  incorporating  both 
kinematic  and  attribute  information  in  the  association  process.  A  BBN  model  for  a  single  scan  data  associa¬ 
tion  problem  is  presented  and  used  to  develop  a  global  nearest  neighbours  solution  using  both  kinematic  and 
attribute  information.  Monte  Carlo  simulations  demonstrate  the  benefit  of  using  attribute  information  in  the 
association  process. 

1  INTRODUCTION 

Situational  awareness  plays  a  major  role  in  many  military  and  civilian  operations.  The  automated  situational 
picture  traditionally  displayed  the  position,  speed  and  heading  of  targets  within  a  region  of  interest,  with  the 
operator  manually  adding  identity  and  other  information.  However,  modern  multi-source  algorithms  have  the 
potential  to  automaticially  update  additional  target  information  or  attributes,  such  as  identity,  class,  category, 
weapons  state,  fuel  status,  threat  level  and  intent,  using  the  kinematic  and  attribute  information  from  a  variety 
of  disparate  sensors  and  sources.  The  Australian  Defence  Organisation  is  currently  acquiring  a  number  of 
capabilities,  such  as  the  Airborne  Early  Warning  and  Control,  Air  Defence  Ground  Environment  and  Joint 
Strike  Fighter  that  will,  or  are  expected  to,  provide  some  level  of  automation  in  this  area. 

The  most  critical  and  difficult  problem  in  state  estimation  or  tracking  problems  is  that  of  data  association. 
The  use  of  attribute  information  in  data  association  has  the  potential  to  provide  greater  discrimination  and 
reduce  association  ambiguities.  This  paper  addresses  the  joint  kinematic  and  attribute  data  association  problem, 
using  a  common  probabilistic  framework  defined  for  both  kinematic  and  attribute  data  using  Bayesian  Belief 
Networks  (BBNs).  This  framework  accomodates  the  uncertainty  inherent  in  all  sensor  data,  which  is  ignored 
in  simple  attribute  gating  techniques. 

The  paper  is  organised  as  follows.  Following  this  introduction,  an  overview  of  relevant  previous  work  is 
presented  in  Section  2,  and  a  brief  tutorial  on  BBNs  is  provided  in  Section  3.  In  Section  4,  the  joint  kinematic 
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and  attribute  data  association  algorithm  is  developed,  with  the  results  of  the  Monte  Carlo  analysis  in  Section  4, 
and  conclusions  in  Section  6. 

2  BACKGROUND 

Tracking  the  dynamic  or  kinematic  state  of  a  target  is  well  documented  [1],  However,  the  dynamic  behaviour  of 
a  target  and  information  from  other  sensors  and  sources,  such  as  secondary  surveillance  radar  (SSR),  electronic 
surveillance  (ES),  tactical  data  links  and  intelligence  sources,  can  be  used  to  estimate  other  attributes  of  a  target, 
such  as  its  identity,  class,  threat  level  and  intent.  Two  key  challenges  are  to  successfully  combine  the  disparate 
information  or  data,  and  to  handle  the  uncertainty  or  errors  in  the  data.  Bayesian  and  Dempster-Shafer  are 
two  approaches  that  are  able  to  accommodate  these  challenges.  Bayesian  techniques  use  Bayesian  probability 
theory  to  handle  uncertainty,  whereas  Dempster-Shafer  introduces  the  concepts  of  support  and  plausibility 
[2,  3].  Bayesian  probability  theory  provides  a  common  frame  of  reference  for  the  kinematic  and  attribute 
variables,  and  has  previously  been  exploited  for  joint  tracking  and  classification  [4,  5]. 

Bayesian  belief  networks  (BBNs)  are  causal  belief  networks  that  model  the  dependencies  between  vari¬ 
ables  or  network  nodes  using  Bayesian  probability  [6].  The  ability  of  BBNs  to  model  both  continuous  and 
discrete  variables,  and  the  relationships  between  such  variables,  make  them  excellent  candidates  for  devel¬ 
oping  Bayesian  approaches  to  attribute  tracking  and  fusion  systems.  Korpisaari  and  Saarinen  [7]  proposed 
a  generalised  Bayesian  network  structure  with  a  many-dimensional  association  vector  for  applying  attribute 
data  in  a  Joint  Probabilistic  Data  Association  (JPDA)  context.  With  Hautaniemi  [8],  these  authors  recognised 
how  the  dependencies  between  attributes  generally  produce  non-singly  connected  networks  for  attributes  that 
evolve  over  time.  Chang  and  Fung  [9]  used  a  BBN  to  incorporate  attribute  information  into  a  multiple  hypoth¬ 
esis  tracking  algorithm.  Krieg  [10]  showed  how  BBNs  may  be  used  as  a  basis  for  deriving  the  Kalman  filter 
tracking  algorithm  and  a  joint  kinematic  and  attribute  nearest  neighbours  data  association  algorithm.  He  also 
developed  an  attribute  estimation  algorithm,  which  he  applied  to  the  target  tracking  and  recognition  application. 
In  [11],  he  used  BBNs  to  develop  a  joint  tracking  and  classification  solution. 

3  OVERVIEW  OF  BAYESIAN  BELIEF  NETWORKS 

A  Bayesian  belief  network  (BBN)  is  a  causal  network  that  is  represented  by  a  directed  acyclic  graph,  where  the 
nodes  represent  stochastic  variables  and  the  directed  links  or  arcs  represent  the  direct  causal  influences  between 
the  variables  [12].  Bayesian  calculus,  in  particular  predetermined  prior  and  conditional  probabilities,  is  used 
to  determine  the  posterior  node  probabilities  from  the  data  or  evidence  applied  to  the  network.  Although  the 
network  provides  a  complete  probabilistic  model  of  all  the  variables  in  a  specific  domain,  its  design  is  based 
on  local  interactions  between  variables  that  directly  influence  one  another.  Therefore,  it  is  only  necessary  to 
determine  the  conditional  probabilities  for  each  variable  [6] ;  a  conceptually  simpler  task  than  defining  the  entire 
joint  probability. 

The  belief  or  posterior  probability  of  the  state  of  each  variable  in  a  BBN  is  dynamically  calculated  from 
the  static  conditional  probabilities  and  the  propagation  of  evidence.  When  applied  to  the  network,  evidence 
is  propagated  in  the  direction  of  the  causal  links  by  n  messages  and  against  the  direction  of  the  links  by  A 
messages.  On  receipt  of  a  message  from  another  node,  a  node  will  update  its  belief  and  propagate  new  messages 
to  each  node  directly  connected  to  it,  thereby  ensuring  the  evidence  is  propagated  throughout  the  network.  The 
belief  at  any  node  is  the  normalised  product  of  the  A  and  n  information  at  that  node. 

For  simplicity,  the  following  description  of  evidence  propagation  in  BBNs  is  restricted  to  tree  network 
structures.  Unless  stated  otherwise,  a  node  or  variable  is  denoted  by  an  uppercase  alphabetic  character,  and  its 
value  is  denoted  by  its  lowercase  equivalent. 

Consider  the  propagation  of  evidence  through  the  BBN  in  Figure  1.  The  evidence  influencing  the  proposi¬ 
tion  X  =  x  is  separated  into  two  disjoint  subsets,  namely  that  introduced  to  X  through  the  arcs  between  X  and  its 
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children,  Y\  and  that  introduced  through  the  arc  between  X  and  its  parent,  W.  The  former  is  aggregated 

into  the  likelihood  A(x),  representing  the  diagnostic  or  retrospective  support  for  the  proposition  X  =  x,  and  the 
latter  the  prior  probability  n{x),  representing  the  causal  or  predictive  support  for  X  =  x.  The  belief  of  X  =  x  is 
then  simply 

Bel(jt)  =  an(x)X(x),  (1) 

where  a  is  a  normalising  constant. 

For  simplicity,  and  without  loss  of  generality,  it  is  assumed  that  all  evidence  is  introduced  into  the  BBN 
through  leaf  nodes.  Then  the  retrospective  support  provided  by  the  evidence  on  each  leaf  node  YJ-,  i  =  1, . . .  ,N, 
is  reflected  by  the  message  A(y,-),  which  takes  the  value  <5v,y,  if  instantiated  with  y\,  or  some  other  value 
representing  the  probability  that  Yj  =  vy  This  support  is  then  propagated  to  X  as 


h  (x)  =  £  p  0;  |  x)  A  (>,•) ,  (2) 

yt 

and  combined  with  the  support  from  the  other  children  of  X  using 

N 

A(x)  =  (3) 

i=l 

The  retrospective  support  is  then  propagated  to  W  according  to 

kx{w)  =  YjY(x\w)X{x).  (4) 

Jt 

The  n  messages  propagate  predictive  support  in  the  direction  of  the  links.  The  support  provided  by  the  root 
node  W  is  simply  the  prior  probability  of  W,  that  is,  n(w)  =  p  (w),  and  this  is  propagated  to  X  using 

n(x)  =  ^p{x\w)k{w).  (5) 

w 
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The  propagation  continues  from  X  to  the  children  of  X  using 

,  \  ni  n  m  Bel(jc 
nYi(x)  =  a  [[  hk{x)n(x)  =  a— — 

k=\\i  AYiW 

and 

n(yd  =  Zp(y«  I  x)ky,(x)- 

X 

The  belief  at  all  nodes  may  now  be  calculated. 

4  JOINT  KINEMATIC  AND  ATTRIBUTE  FUSION 


(6) 

(7) 


Figure  2.  6BN  for  associating  measurements  to  tracks 

Consider  the  problem  of  associating  N  measurements  at  some  time  t  to  M  tracks  or  other  entities.  Denoting 
the  measurements  as  z"  ,  n  =  1,2, . . .  ,N,  and  the  track  states  as  x,,  m  =  1,2, . . . .  M,  and  introducing  an  N  x  I 
association  vector,  kt  =  (k,  ,kf  . .  ,k,'l)T,  k\,]  S  {1,2, .. .  ,M},  where  the  value  of  the  ith  element  indicates 
the  track  to  which  the  z'th  measurement  is  associated,  the  problem  becomes  one  of  estimating  the  value  of  k,. 
This  may  be  represented  by  the  BBN  in  Figure  2.  Both  the  measurements  and  track  states  may  contain  both 
attribute  and  kinematic  information,  and  the  corresponding  probabilities  represent  the  joint  probabilities  of  all 
the  attribute  and  kinematic  elements. 

The  BBN  of  Figure  2  is  not  singly  connected,  that  is,  it  contains  loops  in  the  underlying  Markov  network. 
Therefore,  the  clustering  approach  [6,  13]  is  used  here  to  produce  the  tree  structure  of  Figure  3,  where  the 
clustered  variable  At  =  {X,  ,Xf"\ . . .  ,X,M]  .K,\.  The  conditional  probabilities  are  now  given  as 

P  (4H)  I  A)  =  P  (z(,n)  |  kt,xtk'  ^  ,  n  =  1,2,. ..  ,1V.  (8) 

From  (1),  the  posterior  probability,  or  belief,  of  A,  is  given  by 

Bel(Af)  =  aX{At)n[At).  (9) 

The  predictive  support  is  simply  the  prior  probability  of  At,  namely 

M  ft 

n(At)  =  p  (At)  =  p  {k,)  n  p  [x,j  ,  (10) 

m= 1 
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Figure  3.  Cluster  BBN  for  associating  N  measurements  to  M  tracks 


assuming  that  the  track  prior  probabilites  are  independent,  which  is  valid  if  it  is  assumed  that  a  measurement 
can  only  be  associated  with  a  single  track. 


Denoting  the  known  value  of  zA  as  4”4 


A(4"})  =  8  w >)', 


z,  z, 


(11) 


where  5»»/  equals  1  when  z,"  '  =  zA'  and  0  otherwise.  Using  (2)  and  (8),  the  retrospective  support  to  A,  from 


the  72th  measurement  becomes 


z,  z, 


\{n) {At)  =  p  (zin)'  |  kt,xA  ^  , 


and,  from  (3),  the  support  for  A,  from  all  measurements  is 


(12) 


N 


HAt)= np(4',)'i^' 

n— 1 


(13) 


Invoking  (9)  with  (10)  and  (13)  gives 


M  '7  ^  N 

Am) 


Bel(A,)  =  ap(fc)  PI  P  (4”’)  PIp  (  4"1'  I  fc.U'"1 

m— 1  n— 1 

The  belief  of  kt  may  now  be  found  by  marginalising  A,,  that  is, 


(14) 


M 


(l)<2)  JM)m= 1 


Bel (kt)  =  ap  (kt)  X  X  X  fl  P  (4'"’ )  fT  P  (  1  *u4 


N 


n=l 


,(«)'  i  /,  Jk\n)) 


(15) 


which  may  be  rewritten 


m:mEkt  Am) 

A. 


Bel(k,)  =  ap  (kt)  II  X  P  (4“’ )  II  P  [ZA'  I  4 


1  ( n ) 

— m 


(16) 


where  a  is  a  normalising  constant.  Using  the  most  probable  value  as  the  best  estimate  for  the  association  vector, 
a  may  be  ignored,  as  the  solution  is  simply 


m:mEkt  Am) 

A. 


h  =  argmaxp  (kt)  ]^[  X  P  (4'"' )  II  P  (4";/ I  4 


rr.kj  —m 


( n)/  |  (m) 


(17) 
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Of  particular  interest  is  the  prior  distribution  of  kt.  If  all  permutations  of  the  association  vector  are  equi- 
probable,  then  the  solution  under  linear  and  Gaussian  assumptions  is  equivalent  to  the  nearest  neighbours 
association  algorithm  for  both  kinematic  and  attribute  data.  However,  zeroing  the  prior  probability  of  all  the 
permutations  of  the  association  vector  that  associate  more  than  one  measurement  from  any  sensor  to  one  same 
target  constrains  the  solution  and  the  result  is  a  global  nearest  neighbours  algorithm  for  both  kinematics  and 
attributes. 


5  RESULTS 

The  association  algorithm  of  Section  4,  with  the  constraint  that  each  target  can  only  produce  one  measurement 
from  each  sensor,  is  applied  to  the  tracking  and  identification  problem  for  crossing  targets.  Probabilistic  gating 
is  used  to  eliminate  low  probability  associations.  Measurements  are  simulated  from  two  sensors,  namely  a 
secondary  surveillance  radar  (SSR)  returning  range,  azimuth  and  identity,  and  an  Electronic  Surveillance  (ES) 
sensor  returning  azimuth  and  platform  class,  which  is  related  to  identity.  Kinematic  and  identity  updates  are 
performed  by  the  Extended  Kalman  Filter  (EKF)  and  the  identity  update  algorithm  of  [10],  respectively.  Two 
cases  are  considered,  one  with  association  based  only  on  kinematic  data,  and  the  other  using  both  kinematic 
and  attribute  data  for  association. 

Figures  4  and  5  show  the  track  locations  for  a  single  run  for  kinematic  only  association  and  joint  attribute 
and  kinematic  association,  respectively.  The  track  positions  are  indicated  by  solid  lines,  and  the  ground  truth 
by  dotted  lines.  The  SSR  position  measurements  from  each  target  are  shown,  where  the  *+’  represents  those 
from  one  target,  and  the  ‘o’  those  from  the  other  target. 

Frequent  track  swaps,  as  illustrated  by  Figure  4,  are  observed  when  only  kinematic  data  is  used  for  data 
association.  The  use  of  both  kinematic  and  attribute  data  overcomes  this  problem.  This  is  reflected  in  the 
assocation  accuracy  results  presented  in  Table  1,  where  the  percentage  of  measurements  from  each  target  that 
are  associated  with  each  track  is  presented  for  both  the  SSR  and  ES  sensors.  Perfect  association  would  result 
in  all  measurements  from  target  1  associated  with  track  1 ,  and  all  measurements  from  target  2  associated  with 
track  2.  The  results  for  both  sensors  reflect  the  difficulty  of  associating  reports  to  closely  spaced  entities  using 
only  kinematic  data.  The  additional  discrimination  introduced  by  the  attribute  data  provides  the  observed 
improvement  in  performance  of  the  joint  association  over  the  kinematic  only  association. 

This  example  demonstrates  the  advantage  of  using  both  attribute  and  kinematic  data  for  data  association. 
This  is  equally  applicable  to  other  data  fusion  applications,  where  other  attribute  information  may  be  involved. 

Table  1.  Association  accuracies  for  kinematic  and  joint  kinematic  and  attribute  association 


Sensor 

Target 

Track 

Assoc.  Accuracies  (%) 

Kinematic 

Joint 

1 

1 

86 

99 

SSR 

2 

13 

005 

1 

13 

0-05 

Z 

2 

85 

98 

1 

1 

58 

71 

ES 

2 

41 

27 

1 

42 

28 

z 

2 

57 

71 
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Figure  5.  Track  location  using  joint  kinematic  and  attribute  association 
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6  CONCLUSIONS 

Accurate  data  association  is  a  necessary  requisite  for  the  fusion  of  multi-source  multi-target  information.  A 
joint  kinematic  and  attribute  data  association  algorithm  that  incorporates  data  uncertainty  has  been  developed 
using  a  BBN  framework.  Significant  improvement  in  association  accuracy  over  kinematic  only  association  has 
been  demonstrated  through  simulation. 

REFERENCES 

[1]  S.  Blackman  and  R.  Popoli,  Design  and  Analysis  of  Modern  Tracking  Systems.  Norwood  MA:  Artech 
House,  1999. 

[2]  P.  Bogler,  “Shafer-Dempster  reasoning  with  applications  to  multisensor  target  identification  systems,” 
IEEE  Trans.  Systems,  Man,  and  Cybernetics,  vol.  SMC-17,  no.  6,  pp.  968-77,  1987. 

[3]  M.-A.  Simard,  J.  Couture,  and  E.  Bosse,  “Data  fusion  of  multiple  sensors  attribute  information  for  target 
identity  estimation  using  a  Dempster-Shafer  evidential  combination  algorithm,”  in  Proc.  SPIE,  vol.  2759, 
pp.  577-88,  1996. 

[4]  S.  Challa  and  G.  W.  Pulford,  “Joint  target  tracking  and  classification,”  in  Proc.  5th  bit.  Conf.  on  Radar 
Systems  (Radar  ’99),  (Brest,  France),  1999. 

[5]  N.  Gordon,  S.  Masked,  and  T.  Kirubarajan,  “Efficient  particle  filters  for  joint  tracking  and  classification,” 
in  Signal  and  Data  Processing  of  Small  Targets  2002,  vol.  4728  of  Proceedings  of  the  SPIE,  pp.  439-49, 
2002. 

[6]  J.  Pearl,  Probabilistic  Reasoning  in  Intelligent  Systems:  Networks  of  Plausible  Inference.  San  Francisco 
CA:  Morgan  Kaufmann,  1988. 

[7]  P.  Koipisaari  and  J.  Saarinen,  “Bayesian  networks  for  target  identification  and  attribute  fusion  with  JPDA ,” 
in  Proc.  2nd  Int.  Conf.  Information  Fusion,  vol.  II,  pp.  763-9,  1999. 

[8]  S.  Hautaniemi,  P.  Korpisaari,  and  J.  Saarinen,  “Target  identification  with  Bayesian  networks,”  in  Proc. 
SPIE,  vol.  4051,  pp.  55-66,  2000. 

[9]  K.-C.  Chang  and  R.  Fung,  “Target  identification  with  Bayesian  networks  in  a  multiple  hypothesis  tracking 
system,”  Optical  Engineering,  vol.  36,  no.  3,  pp.  684-91,  1997. 

[10]  M.  F.  Krieg,  “A  Bayesian  belief  network  approach  to  multi-sensor  kinimatic  and  attribute  tracking,”  in 
Proc.  Information,  Decision  and  Control  (IDC2002),  (Adelaide,  Australia),  Feb.  2002. 

[11]  M.  L.  Krieg,  “Joint  multi-sensor  kinematic  and  attribute  tracking  using  bayesian  belief  networks,”  in  Proc. 
Int.  Conf.  Information  Fusion  (Fusion  2003),  (Cairns  Australia),  July  2003. 

[12]  F.  Jensen,  An  Introduction  to  Bayesian  Networks.  Fondon  UK:  UCF  Press,  1996. 

[13]  M.  L.  Krieg,  “A  tutorial  on  Bayesian  belief  networks,”  Tech.  Rep.  DSTO-TN-0403,  DSTO  Australia, 
2001. 


15-8 


RTO-MP-IST-040 


Joint  Multi-Sensor  Kinematic  and 
Attribute  Tracking  Using  Bayesian 

Belief  Networks 

Mark  L.  Krieg 

mark . krieg@dsto . defence . gov . au 

Intelligence,  Surveillance  and  Reconnaissance  Division 
Defence  Science  and  Technology  Organisation,  Australia 


Overview 


-  Problem 

-  Bayesian  Belief  Networks 

-  Algorithm  Development 

-  Results 

-  Conclusions 


Problem 


-  Attribute  data  or  information 

-  Multiple  disparate  sensors  and  sources 

-  Contribution  to  automated  situational  picture 

-  Data  association 

-  Considered  most  challenging  problem 

-  Attributes  may  provide  greater  discrimination 

-  Need  to  consider  uncertainty 


Bayesian  Belief  Networks 

Causal  network 

-  Continuous  and  discrete  variables 

-  Bayesian  calculus 

-  Uncertainty 

-  Complete  probabilistic  model  of  all  variables 

-  Only  need  to  define  local  interactions 

-  Evidence  propagation  through  network 


Evidence  Propagation  in  a  BBN 


-Vi^)  s 


X(Vi)  Xy,  y'- 

xyAx)  =  ^2?(yi  I  x)KVi) 

Vi 


A(x)  =  Ay1(x)Ay2(x) 

A (w)  =  p  (x  |  w)  X(x) 


Evidence  Propagation  in  a  BBN 


7r(w)  =  p  (w) 

7t(x)  ~  p  (x  |  w)  7r(w) 


w 


7Ty1(x)  =  \y2(x)tt(x) 

n{yi)  =  | 


Algorithm  Development 
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Associating  Measurements  to  Tracks 


Problem:  Loops  in  underlying  Markov  network 


Associating  Measurements  to  Tracks 


Algorithm 

Posterior  Probability: 


Bel(fct)  =  ap  (kt)  n  Ep(^) 

mn'.mn^kt  ~(m) 

x t 


kt  =  argmaxBel(^) 

h 


Best  estimate: 
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Results 


Simulations 


-  Two  targets  crossing 

-  Two  sensors: 

-  Secondary  Surveillance  Radar 

-  Electronic  Support  (ES) 

-  Association  constrained  to  one 
measurement  /  target  /  sensor 

-  Probabilistic  gating  applied  prior  to  association 

-  Kinematic  state  update  using  EKF 

-  ID  update  using  algorithm  of  [Krieg,  IDC2002] 

-  Simulations  for  both  kinematic  only  and  joint 
kinematic  /  attribute  association 
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Track  Position  Estimates 
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Association  Accuracies 


Sensor 

Target 

Assoc.  Accuracies  (%) 

Kinematic 

Joint 

SSR 

1 

86 

99 

2 

85 

98 

ES 

1 

58 

71 

2 

57 

71 

Conclusions 


-  Accurate  data  association  prerequisite  for  fusion. 

-  Developed  joint  kinematic  &  attribute  association 
algorithm  that: 

-  is  based  on  BBN  framework; 

-  incorporates  uncertainty; 

-  improves  accuracy  over  kinematic  only. 

-  May  be  applied  to  higher  level  applications. 


